Unsupervised Image Clustering using Probabilistic Continuous Models and Information Theoretic Principles

This thesis proposes a new method for unsupervised image clustering using probabilistic continuous models and information theoretic principles. Image clustering relates to content-based image retrieval systems. It enables the implementation of efficient retrieval algorithms and the creation of a user friendly interface to the database. The thesis presents a coherent theory for continuous probabilistic image modeling based on mixture of Gaussians densities. The continuous image modeling is extended to the modeling of an image-set created by a supervised or an unsupervised clustering process. Three ways of obtaining the image-set model are introduced and the difference between them is discussed. Supervised image-set (category) modeling is utilized to compare between the proposed continuous models and the more traditional discrete image modeling based on histograms. The unsupervised image clustering framework is based on a continuous version of a recently introduced information theoretic principle, the information bottleneck (IB). The clustering method is based on hierarchical grouping: Utilizing a Gaussian mixture model (GMM), each image in a given archive is first represented as a set of coherent regions in a selected feature space. Images are next grouped enabling the mutual information between the clusters and the image content to be maximally preserved. The appropriate number of clusters can be determined directly from the IB principle. Several clustering algorithms based on the IB principle are presented, including a comparison between them. An incremental method for calculating mutual information between images and their continuous representation, which is a byproduct of the IB principle, is introduced. Mutual information is then used as a quality measure for estimating image representations and clustering quality. Experimental results demonstrate the performance of the proposed clustering methods on a real image database. The influence of image representation on the clustering results is also discussed.